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(54) Identification data insertion and detection system for digital data 



(57) A identification data insertion system for digital 
data includes a DOT calculator (103) for extracting 
blocks (102) of 8 x 8 pixels from within a current image 
(101) and performing DCT (discrete cosine transform) 
calculation for the blocks (102), a quantizer (104) for 
quantizing data outputted from the DCT calculator 
(103), a quantization table (105) for being referred to 
when the quantizer (104) performs quantization, an 
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electronic watermark data insertion element (107) for 
inserting electronic watermark data (106) into data after 
quantization, and an encoder (113) for encoding data 
outputted from the electronic watermark data insertion 
element (1 07) to produce MPEG data (1 1 4) in which the 
electronic watermark data (106) are inserted and out- 
putting the MPEG data (114). 
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Description 

This invention relates to a system for processing a 
digital image, and more particularly to a system for 
embedding electronic watermark data as identification 5 
data having special information into a digital image. 

In recent years, illegal duplication of a digital image 
has become a serious problem. 

In order to prevent illegal duplication, a system has 
been proposed wherein digital image data are ciphered 10 
such that only a reproduction system having a legal 
decipherment key can reproduce the ciphered digital 
image data. However, once the cipher is deciphered, 
later duplication cannot be prevented. 

Thus, in order to prevent illegal use and duplication is 
of a digital image, a method has been proposed wherein 
electronic watermark data which are special information 
are embedded into the digital image itself. 

As such electronic watermark data for a digital 
image as described above, two kinds of electronic 20 
watermark data of visible electronic watermark data and 
invisible electronic watermark data are available. 

The visible electronic watermark data are used 
such that a special character or symbol or the like is 
combined with an image so that it may be visually rec- 25 
ognized. While the visible electronic watermark data 
deteriorate the picture quality, they have an effect of vis- 
ually appearing to a user of a digital image for preven- 
tion of illegal use of the digital image. 

An example of embedding of visible electronic 30 
watermark data is disclosed in Japanese Patent Laid- 
Open Application No. Heisei 8-241403. According to the 
method disclosed in the document just mentioned, visi- 
ble electronic watermark data are combined with an 
original image such that only the brightness compo- 35 
nents of those pixels which correspond to an opaque 
portion of the electronic watermark data are varied 
while color components of the pixels are not varied. In 
this instance, the scaling value by which the brightness 
components of pixels are varied is determined based on 40 
color components, random numbers, values of pixels of 
the electronic watermark data and so forth. 

On the other hand, the invisible electronic water- 
mark data are embedded into an image so that the pic- 
ture quality may not be deteriorated. Since the invisible 45 
electronic watermark data little deteriorate the picture 
quality, they are characterized in that they cannot be 
recognized visually. 

However, if special information which allows identi- 
fication of an author is embedded as the electronic so 
watermark data, then also after illegal duplication is per- 
formed, the author can be specified by detecting the 
electronic watermark data. On the other hand, if dupli- 
cation inhibition information is embedded, then, for 
example, when a reproduction apparatus detects the 55 
duplication inhibition information, it is possible to notify 
the user that duplication of the data is inhibited or to 
render a duplication prevention mechanism in the repro- 
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duction apparatus operative to limit duplication of the 
data by a video tape recorder (VTR) or the like. 

One available methods of embedding invisible elec- 
tronic watermark data into a digital image is to embed 
special information as electronic watermark data into a 
portion of pixel data such as a least significant bit (LSB) 
which has a comparatively little influence on the picture 
quality. However, where this method is applied, elec- 
tronic watermark data can be removed readily from an 
image. For example, if a low-pass filter is used, then 
information of the LSB of a pixel is lost. Further, elec- 
tronic watermark data are lost also by image compres- 
sion processing since the image compression 
processing reduces the data amount by removing infor- 
mation at such a portion which has a comparatively little 
influence on the picture quality as mentioned above. 
Consequently, it is difficult to redetect the electronic 
watermark data. 

Another method is disclosed in Japanese Patent 
Laid-Open Application No. Heisei 6-315131. 

In the method of the document just mentioned, a 
region which can be replaced, upon reproduction, by a 
peripheral region without causing deterioration of an 
image is detected using a correlation of images of suc- 
cessive frames, and the level of the region of an object 
of conversion is converted to embed particular informa- 
tion. In this method, upon reproduction, the region in 
which identification data are embedded is specified 
using the signal missing portion and the replacement 
information, and the portion is corrected to restore the 
image. 

A further method is disclosed in Japanese Patent 
Laid-Open Application No. Heisei 5-30466 wherein a 
video signal is frequency converted and information 
having a frequency signal lower than a frequency band 
of the video signal after the frequency conversion. In 
this method, the original video signal is extracted using 
a high pass filter while embedded identification data are 
extracted using a low pass filter. 

As another method which involves frequency con- 
version of an image, a method has been proposed 
wherein an image is frequency converted and electronic 
watermark data are embedded into a region of the video 
signal after the frequency conversion in which strong 
frequency components are included (Nikkei Electronics, 
April 22, 1996, No. 660, p.13). 

In this method, since electronic watermark data are 
embedded into frequency components, the electronic 
watermark data are not lost even by image processing 
such as compression processing or filtering. Further, 
random numbers which exhibit a normal distribution are 
adopted as the electronic watermark data to prevent 
interference between electronic watermark data and 
make it difficult to destroy electronic watermark data 
without having a significant influence on the entire 
image. 

According to the electronic watermark data embed- 
ding method in the method described above, an original 
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image 501 is converted into frequency components 
using a DCT (discrete cosine transform) transformer 
502 as seen in FIG. 5. Then, n data which exhibit com- 
paratively high values in a frequency region are 

selected and individually represented by f(1), f(2) 5 

f(n), and electronic watermark data 503 (w(1), w(2), .... 
w(n)) are selected from within a normal distribution 
whose average is 0 and whose dispersion is 1 . Then, 
F(i) = f(i) + a x |f(i)| x w(i) is calculated for each i by an 
electronic watermark data insertion element 504. Here, w 
a is a scaling factor. Finally, DCT transform coefficients 
including those frequency components in which f(i) are 
replaced by F(i) are inverse transformed by an inverse 
DCT transformer 509 to obtain an image 51 0 in which 
the electronic watermark data is embedded. 15 

Electronic watermark data is detected using the fol- 
lowing method. In the detection method, an original 
image and electronic watermark data candidates w(i) 
(where i = 1 , 2 n) must be known. 

Referring to FIG. 6, an original image 601 and an 20 
electronic watermark data containing image 602 are 
converted into frequency components using DCT trans- 
formers 603 and 604, respectively. In a frequency 

region, values of factors corresponding to f(1) ( f(2) 

f(n) in which electronic watermark data are embedded 25 

are represented by F(1), F(2) F(n), respectively. An 

electronic watermark data extractor 605 extracts elec- 
tronic watermark data W(i) based on f (i) and F(i) by cal- 
culation of W(i) = (F(i) - f(i))/f(i) . Then, an inner product 
calculator 608 calculates, based on electronic water- 30 
mark data 609, a statistical similarity between w(i) and 
W(i) in accordance with C = Wx w/(WD x wD) using 
inner products of vectors. Here, W = (W(1), W(2), .... 

W(n)), w = (w(1), w(2) w(n)), WD = absolute value of 

the vector W, and wD = absolute value of the vector w. 35 
A statistical similarity discriminator 610 discriminates, 
when the statistical similarity C is higher than a particu- 
lar value, that the pertaining electronic watermark data 
are embedded. 

If electronic watermark data are embedded in an 40 
image using the method described above, then they can 
be effectively used when an author who owns the origi- 
nal image performs detection processing for digital 
image data which are supposed to be an illegal dupli- 
cate. 45 

However, since the method described above 
requires an original image, although it is effective when 
an author who owns an original image performs detec- 
tion processing for image data which are supposed to 
be an illegal duplicate, a reproduction apparatus of a so 
terminal cannot perform electronic watermark data 
detection processing because it does not have the orig- 
inal image. Thus, another method which is an improve- 
ment over the method described above and is directed 
to terminal processing and particularly to an MPEG ss 
(Moving Picture Experts Group) system has been pro- 
posed. 

According to the improved method, an original 



image is divided into blocks of 8 pixels x 8 pixels, and 
embedding and extraction processing of electronic 
watermark data is performed regarding each block as a 
processing unit. 

In electronic watermark data embedding process- 
ing, data are first set successively as f(1), f(2), .... f(n) in 
the ascending order of the frequency component of an 
AC component in a frequency region after discrete 
cosine transform in MPEG coding processing is com- 
pleted, and electronic watermark data w(1), w(2) 

2(n) are selected from within a normal distribution 
whose average is 0 and whose dispersion is 1 , and then 
F(i) = f(i) + a x avg(f(i)) x w(i) is performed for each i. 
Here, a is a scaling factor, and avg(f(i)) is a partial aver- 
age which is an average of absolute values of three 
neighboring points to f(i). Then, succeeding processing 
of MPEG coding processing is performed replacing f(i) 
with F(i). 

Detection of electronic watermark data is per- 
formed in accordance with the following method. In this 
detection method, an original image is not required, but 
only it is required that data candidates w(i) (where i = 1 , 
2 n) be known. 

In a frequency region of each block after dequanti- 
zation of MPEG decompression processing is com- 
pleted, data are represented as F(1), F(2) F(n) in the 

ascending order of the frequency component. An aver- 
age value of absolute values of three neighboring points 
to F(i) is represented as a partial average avg(F(i)), and 
electronic watermark data W(i) is calculated in accord- 
ance with W(i) = F(i)/avg(F(i)) . Further, a sum total 
WF(i) of W(i) for one image is calculated for each i. 
Thereafter, a statistical similarity between w(i) and 
WF(i) is calculated in accordance with 
C = WF x w/(WFD x wD) making use of an inner prod- 
uct of vectors. Here, W « (WF(1), WF(2), .... WF(n)), w 

= (w(1), w(2) w(n)), WFD = absolute value of the 

vector WF, and wD = absolute value of the vector w. 
When the statistical similarity C has a value higher than 
a particular value, it is discriminated that the electronic 
watermark data are embedded. 

FIG. 7 shows a conventional system which embeds 
electronic watermark data using such a method as 
described above, and FIG. 8 shows a conventional sys- 
tem which detects electronic watermark data embed- 
ded by such a system as shown in FIG. 7. 

Referring first to FIG. 7, in the system shown, a cur- 
rent image 701 is divided into blocks 702 of 8 x 8 pixels 
and discrete cosine transformed by a DCT transformer 
703, and then electronic watermark data 704 are 
inserted into an output of the DCT transformer 703 by 
an electronic watermark data insertion element 705. In 
the electronic watermark data insertion element 705. 
partial averages mentioned above are calculated by a 
partial average calculator 706. and a result of the calcu- 
lation and the electronic watermark data 704 are multi- 
plied by a multiplier 708. Then, an output of the 
multiplier 708 is multiplied by a constant (scaling factor) 
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a 707 by another multiplier 709. Then, a result of the 
multiplication is added to the image data from the DCT 
transformer 703 by an adder 710 to make image data in 
which the electronic watermark data are embedded. 
The image data are quantized by a quantizer 71 1 with 5 
reference to a quantization table 712 and then encoded 
by an encoder 71 3 to produce MPEG image data 71 4 in 
which the electronic watermark data are embedded, 
and the MPEG image data 714 are outputted. 

Referring now to FIG. 8, in the system shown, 
MPEG image data 801 in which electronic watermark 
data are embedded are decoded by a decoder 802 and 
then dequantized by a dequantizer 803, whereafter they 
are inverse DCT transformed by an inverse DCT calcu- 
lator 804 to restore original image data 805. Meanwhile, 
electronic watermark data are extracted from the output 
of the dequantizer 803 by an electronic watermark data 
extractor 806. In the electronic watermark data extractor 
806, after partial average calculation by a partial aver- 
age calculator 807 and division by a divider 808 are per- 
formed, sum totals WF(i) for one screen are performed 
for each i by an adder 809. Then, inner products of the 
sum totals W(i) and electronic watermark data 81 1 of an 
object of detection are calculated by an inner product 
calculator 81 0 to obtain a statistical similarity 812. Then, 
if the statistical similarity 812 has a value higher than a 
particular value, then it is discriminated that the elec- 
tronic watermark data are embedded in the MPEG 
image data 801. 

In the method disclosed in Japanese Patent Laid- 
Open Application No. Heisei 6-315131, since electronic 
watermark information is not embedded in all frames, 
illegal duplication cannot be prevented with regard to 
any frame in which the electronic watermark information 
is not embedded. Further, since successive frames rep- 
resent still pictures and it is presupposed that there is no 
variation between successive frames, where moving 
pictures which involve strong movements are handled, a 
region into which electronic watermark data are to be 
embedded cannot be specified. Consequently, elec- 
tronic watermark cannot be embedded. 

Meanwhile, in the method disclosed in Japanese 
Patent Laid-Open No. Heisei 5-30466, since electronic 
watermark data are embedded into a portion of an 
image which is lower than a frequency region after fre- 
quency conversion of the image, the electronic water- 
mark data can be removed readily using a band-pass 
filter. 

Further, with the method wherein electronic water- 
mark data are embedded into a portion having a strong 
frequency component after frequency conversion, while 
electronic watermark data cannot be removed using a 
filter or the like, electronic watermark data inserted after 
DCT processing are sometimes lost by quantization 
processing. This is because, where a value of any fre- 
quency component after DCT falls within a fixed range, 
it is converted into a particular representative value in 
quantization processing. 
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Accordingly, due to an action of quantization, the 
values of a partial average avg(f(i)) of three neighboring 
points upon insertion and a partial average avg(F(i)) of 
the three neighboring points upon detection are some- 
times different by a large amount. In such an instance, 
the statistical similarity between extracted electronic 
watermark data and electronic watermark data to be 
detected exhibits a low value and makes a factor which 
causes an erroneous discrimination result. 

It is an object of the present invention to provide an 
identification data insertion and detection system for 
digital data which improves the detection ratio when an 
image is converted into frequency components and 
electronic watermark data are embedded in a portion of 
an image having a strong frequency component. 

It is another object of the present invention to pro- 
vide an identification data insertion and detection sys- 
tem for digital data which allows a statistical similarly to 
be calculated with a higher degree of accuracy. 

In order to attain the objects described above, 
according to the present invention, taking it into consid- 
eration that DCT coefficients are rounded by quantiza- 
tion, insertion of electronic watermark data is performed 
not in a stage between DCT processing and quantiza- 
tion processing, but after quantization. On the other 
hand, upon detection, electronic watermark data detec- 
tion processing is performed prior to dequantization. 

In the meantime, where insertion of electronic 
watermark data is performed in a stage between DCT 
processing and quantization processing similarly as in 
the conventional methods, when a partial average is to 
be calculated, an absolute value of a value obtained by 
multiplying an integral value obtained by division of a 
corresponding value of a quantization further by the cor- 
responding value of the quantization table. 

More particularly, according to an aspect of the 
present invention, there is provided an identification 
data insertion system for digital data for inserting elec- 
tronic watermark data as identification data having spe- 
cial information into a digital image, comprising a DCT 
calculator for extracting blocks of 8 x 8 pixels from a cur- 
rent image and performing discrete cosine transform 
calculation for the blocks, a quantizer for quantizing data 
outputted from the DCT calculator, a quantization table 
for being referred to when the quantizer performs quan- 
tization, an electronic watermark data insertion element 
for inserting electronic watermark data into the data 
after the quantization, and an encoder for encoding data 
outputted from the electronic watermark data insertion 
element. 

The electronic watermark data insertion element 
may include a partial average calculator for calculating 
an average of absolute values of three neighboring 
points of each factor of the data outputted from the 
quantizer as a partial average, a first multiplier for multi- 
plying the electronic watermark data by an output of the 
partial average calculator for the individual factors, a 
second multiplier for multiplying each factor of an output 
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of the first multiplier by a constant to be used to vary the 
size of the electronic watermark data to be inserted, and 
an adder for adding an output of the second multiplier 
and the output of the quantizer for the individual factors. 

According to another aspect of the present inven- s 
tion, there is provided an identification data detection 
system for digital data adapted to detect electronic 
watermark data inserted by the identification data inser- 
tion system for digital data described above and com- 
prising a decoder for decoding compressed image data, 
a dequantizer for dequantizing data outputted from the 
decoder, an inverse DCT calculator for performing 
inverse discrete cosine transform calculation for data 
outputted from the dequantizer and outputting image 
data obtained by the inverse discrete cosine transform 
calculation, an electronic watermark data extractor for 
extracting and outputting, from among the data output- 
ted from the decoder, those data which are supposed to 
be electronic watermark data, an adder for adding data 
in units of a block of 8 x 8 pixels outputted from the elec- 
tronic watermark data extractor for individual factors 
over one screen, and an inner product calculator for cal- 
culating inner products of electronic watermark data to 
be detected and the data outputted from the adder to 
calculate a statistical similarity and outputting the statis- 
tical similarity. 

The electronic watermark data extractor may 
include a partial average calculator for calculating an 
average of absolute values of three neighboring points 
of each of data in units of a block of 8 x 8 pixels output- 
ted from the decoder as a partial average, and a divider 
for dividing the data outputted from the decoder by the 
partial average outputted from the partial average cal- 
culator. 

With the identification data insertion system and the 
identification data detection system for digital data, 
since electronic watermark data insertion processing is 
performed after quantization whereas electronic water- 
mark data detection processing is performed prior to 
dequantization, otherwise possible loss or disappear- 
ance of electronic watermark data by quantization can 
be prevented, and the difference between a partial aver- 
age used upon insertion and another partial average 
used upon detection is reduced. Consequently, a statis- 
tical similarity can be calculated with a higher degree of 
accuracy. Accordingly, the accuracy in detection of elec- 
tronic watermark data is improved. 

It is to be noted that the identification data insertion 
system and the identification data detection system for 
digital data may be combined so as to form a single 
identification data insertion and detection system for 
digital data. 

According to a further aspect of the present inven- 
tion, there is provided an identification data insertion 
system for digital data, comprising a DCT calculator for 
extracting blocks of 8 x 8 pixels from a current image 
and performing discrete cosine transform calculation for 
the blocks, an electronic watermark data insertion ele- 
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ment for inserting electronic watermark data into data 
after the discrete cosine transform outputted from the 
DCT calculator, a quantizer for quantizing data output- 
ted from the electronic watermark data insertion ele- 
ment, a quantization table for being referred to when the 
quantizer performs quantization and when the elec- 
tronic watermark data insertion element calculates a 
partial average, and an encoder for encoding data out- 
putted from the quantizer. 

The electronic watermark data insertion element 
may include a partial average calculator for calculating, 
using the quantization table when a partial average of 
the data outputted from the DCT calculator is to be cal- 
culated, an average among three neighboring points of 
absolute values of values obtained each by multiplying 
an integer value obtained by dividing a value of each 
factor outputted from the DCT calculator by a corre- 
sponding value of the quantization table by the corre- 
sponding value of the quantization table as a partial 
average of the pertaining factor, a first multiplier for mul- 
tiplying an output of the partial average calculator by the 
electronic watermark data for the individual factors, a 
second multiplier for multiplying data outputted from the 
first multiplier by a constant to be used to vary the size 
of the electronic watermark data to be embedded, and 
an adder for adding the data outputted from the DCT 
calculator and data outputted from the second multiplier 
for the individual factors. 

According to a still further aspect of the present 
invention, there is provided an identification data detec- 
tion system for digital data adapted to detect electronic 
watermark data inserted by the identification data inser- 
tion system for digital data described above and com- 
prising a decoder for decoding compressed image data, 
a dequantizer for dequantizing data outputted from the 
decoder, an inverse DCT calculator for performing 
inverse discrete cosine transform calculation for data 
outputted from the dequantizer and outputting image 
data obtained by the inverse discrete cosine transform 
calculation, an electronic watermark data extractor for 
extracting and outputting, from among the data output- 
ted from the dequantizer, those data which are sup- 
posed to be electronic watermark data, an adder for 
adding data in units of a block of 8 x 8 pixels outputted 
from the electronic watermark data extractor for individ- 
ual factors over one screen, and an inner product calcu- 
lator for calculating inner products of electronic 
watermark data to be detected and the data outputted 
from the adder to calculate a statistical similarity and 
outputting the statistical similarity. 

The electronic watermark data extractor may 
include a partial average calculator for calculating an 
average of absolute values of three neighboring points 
of each of data in units of a block of 8 x 8 pixels output- 
ted from the dequantizer as a partial average, and a 
divider for dividing the data outputted from the dequan- 
tizer by the partial average outputted from the partial 
average calculator. 
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With the identification data insertion system 
wherein insertion of electronic watermark data is per- 
formed in a stage between DCT processing and quanti- 
zation processing similarly as in the conventional 
methods and with the identification data detection sys- 
tem, when a partial average is to be calculated, an 
absolute value of a value obtained by multiplying an 
integral value obtained by division of a corresponding 
value of a quantization further by the corresponding 
value of the quantization table. Consequently the differ- 
ence between a partial average used upon insertion 
and another partial average used upon detection is sup- 
pressed and a statistical similarity can be calculated 
with a higher degree of accuracy. Accordingly, the accu- 
racy in detection of electronic watermark data is 
improved. 

It is to be noted that the identification data insertion 
system and the identification data detection system for 
digital data may be combined so as to form a single 
identification data insertion and detection system for 
digital data. 

The above and other objects, features and advan- 
tages of the present invention will become apparent 
from the following description and the appended claims, 
taken in conjunction with the accompanying drawings in 
which like parts or elements are denoted by like refer- 
ence character. 

FIG. 1 is a block diagram of an electronic water- 
mark data insertion system to which the present 
invention is applied; 

FIG. 2 is a block diagram of an electronic water- 
mark data detection system to which the present 
invention is applied; 

FIG. 3 is a block diagram of another electronic 
watermark data insertion system to which the 
present invention is applied; 
FIG. 4 is a block diagram of another electronic 
watermaik data detection system to which the 
present invention is applied; 
FIG. 5 is a block diagram showing a conventional 
electronic watermark data insertion system; 
FIG. 6 is a block diagram showing a conventional 
electronic watermark data detection system; 
FIG. 7 is a block diagram showing another conven- 
tional electronic watermark data insertion system; 
and 

FIG. 8 is a block diagram showing another conven- 
tional electronic watermark data detection system. 

Referring first to FIG. 1, there is shown an elec- 
tronic watermark data insertion system to which the 
present invention is applied. The electronic watermark 
data insertion system shown includes a DCT calculator 
103 for extracting blocks 102 of 8 x 8 pixels from within 
a current image 101 which makes an object of insertion 
of electronic watermark data and performing DCT (dis- 
crete cosine transform) calculation for the blocks 102, a 



quantizer 104 for quantizing data outputted from the 
DCT calculator 103, a quantization table 105 for being 
referred to when the quantizer 104 performs quantiza- 
tion, an electronic watermark data insertion element 

s 107 for inserting electronic watermark data 106 into 
data after quantization, and an encoder 113 for encod- 
ing data outputted from the electronic watermark data 
insertion element 107 to produce MPEG data 114 in 
which the electronic watermark data 106 are inserted 

io and outputting the MPEG data 1 1 4. 

The electronic watermark data insertion element 
107 includes a martial average calculator 108 for calcu- 
lating a partial average of three neighboring points of 
data outputted from the quantizer 104 in accordance 

is with avg(f(i)) = (|f(i-1)| + |f(i)|+ |f(i+1)|)/3, a multiplier 
110 for multiplying outputs of the electronic watermark 
data 106 and the partial average calculator 108 for indi- 
vidual factors, another multiplier 111 for multiplying 
each factor of an output of the multiplier 1 10 by a con- 

20 stant 1 09 which is used to vary the size of the electronic 
watermark data to be inserted, and an adder 112 for 
adding an output of the multiplier 1 1 1 and the output of 
the quantizer 104 for individual factors. 

The electronic watermark data insertion element 

25 107 performs calculation defined by the following 
expression: 

F(i) = f(i) + avg(f(0) x w(i) x constant 

30 where f(i) is the data outputted from the quantizer 104, 
avg(f(i)) is the data outputted from the partial average 
calculator 108, w(i) is the electronic watermark data 
106, the constant is the constant 109, and i is the 
number of each factor after zigzag scanning of a block 

35 of 8 x 8 pixels. 

Referring now to FIG. 2, there is shown an elec- 
tronic watermark data detection system to which the 
present invention is applied. The electronic watermark 
data detection system detects electronic watermark 

40 data inserted in an image by the electronic watermark 
data insertion system of FIG. 1 described above. The 
electronic watermark data detection system includes a 
decoder 202 for decoding MPEG data 201 which make 
an object of detection of electronic watermark data, a 

45 dequantizer 203 for dequantizing data outputted from 
the decoder 202, an inverse DCT calculator 204 for per- 
forming inverse DCT calculation for data outputted from 
the dequantizer 203 to obtain image data 205 and out- 
putting the image data 205, an electronic watermark 

so data extractor 206 for extracting, from among data out- 
putted from the decoder 202, those data which are sup- 
posed to be electronic watermark data, an adder 209 for 
adding data in units of a block of 8 x 8 pixels outputted 
from the electronic watermark data extractor 206 for 

55 individual factors over one screen, and an inner product 
calculator 210 for calculating inner products of elec- 
tronic watermark data 21 1 to be detected and data out- 
putted from the adder 209 to obtain a statistical 
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similarity 212 and outputting the statistical similarity 
212. 

The electronic watermark data extractor 206 
includes a partial average calculator 207 for calculat- 
ing a partial average of three neighboring points of 5 
data in units of a block of 8 x 8 pixels outputted 
from the decoder 202 in accordance with 
avg(F(i)) = (|F(i-1)| + |F(i)| + |F(i+1)|)/3, and a divider 
208 for dividing data F(i) outputted from the decoder 
202 by a partial average avg(F(i)) outputted from the 10 
partial average calculator 207. 

The electronic watermark data extractor 206 per- 
forms calculation defined by the following expression: 

W(i) = F(i)/avg(F(i)) 15 

where F(i) is the data outputted from the decoder 202, 
avg(F(i)) is the partial average outputted from the partial 
average calculator 207, W(i) is the data which are sup- 
posed to include electronic watermark data outputted 20 
from the electronic watermark data extractor 206, and i 
is the number of each factor of a block of 8 x 8 pixels. 

In the following, operation for insertion and detec- 
tion of electronic watermark data of the electronic water- 
mark data insertion and detection systems described 25 
above with reference to FIGS. 1 and 2 is described in 
detail with reference to FIGS. 1 and 2. 

First, operation for insertion of electronic watermark 
data is described. 

A current image 1 01 is extracted for each block of 8 30 
x 8 pixels by ordinary MPEG compression processing, 
and the DCT calculator 103 performs DCT calculation 
for the thus extracted data. Data of frequency compo- 
nents obtained by the DCT calculation of the DCT cal- 
culator 103 are quantized with reference to the 35 
quantization table 105 by the quantizer 104. The elec- 
tronic watermark data insertion element 107 embeds 
electronic watermark data 106 into the data outputted 
from the quantizer 104. In this instance, the electronic 
watermark data insertion element 107 performs 40 
processing equivalent to calculation of the following 
expression: 

F(i) = f(i) + avg(f (i)) x w(i) x constant 

45 

where i is the number of a factor of a frequency compo- 
nent of the block of 8 x 8 pixels, w(i) is the value of each 
factor of the electronic watermark data 106, the con- 
stant is the constant 1 09, f (i) is each of the factors of the 
8x8 pixel block, avg(f(i)) is the average of absolute val- so 
ues of three neighboring factors, that is, 
(|f(i-1)| + |f(i)| + |f(i+1)|)/3 , and F(i) is the new frequency 
component in which the electronic watermark data 106 
are inserted. 

The encoder 1 13 encodes the data F(i) outputted ss 
from the electronic watermark data insertion element 
1 07 by MPEG processing to produce MPEG data 1 14 in 
which the electronic watermark data 106 are embed- 
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ded. 

Now, operation for detection of electronic water- 
mark data is described. 

First, MPEG data 201 which make an object of 
detection is subject to MPEG decoding by the decoder 
202. The data decoded by the decoder 202 are dequan- 
tized by ordinary MPEG decoding processing by the 
dequantizer 203 and inverse DCT calculated by the 
inverse DCT calculator 204 to produce reproduction 
image data 205. 

The electronic watermark data extractor 206 
extracts the data decoded by the decoder 202, performs 
calculation of F(i)/avg(F(i)) and extracts those data 
which are supposed to be embedded electronic water- 
mark data. Here, F(i) is the frequency component of a 
decoded block of 8 x 8 pixels, and avg(F(i)) is the partial 
average ((|F(i-1)| + |F(i)| + |F(i+1)|)/3 . The adder 209 
calculates a sum total of the extracted data in units of 8 
x 8 pixels outputted from the electronic watermark data 
extractor 206 for individual factors over one screen. 
After the sum totals of the extracted data of the one 
screen are calculated, the inner product calculator 210 
calculates inner products of the sum totals and elec- 
tronic watermark data 211 to be detected to produce a 
statistical similarity 212 and outputs the statistical simi- 
larity 212. 

If the statistical similarity 21 2 is higher than a partic- 
ular fixed value, then it is discriminated that electronic 
watermark data equivalent to the electronic watermark 
data 21 1 are embedded in the MPEG data 201 . 

In this instance, if the electronic watermark data 
211 represent inhibition of duplication, then the repro- 
duction apparatus in which the electronic watermark 
data detection system is incorporated can take such a 
countermeasure as to inhibit duplication of the image 
data 205 produced therein. 

Referring now to FIG. 3, there is shown another 
electronic watermark data Insertion system to which the 
present invention is applied. The electronic watermark 
data insertion system includes a DCT calculator 303 
extracting blocks 302 of 8 x 8 pixels from within a current 
image 301 which makes an object of insertion of elec- 
tronic watermark data and performing DCT calculation 
for the blocks 302, an electronic watermark data inser- 
tion element 305 for inserting electronic watermark data 

304 into data for the DCT transform has been per- 
formed, a quantizer 311 for quantizing data outputted 
from the electronic watermark data insertion element 
305, a quantization table 312 for being referred to when 
the quantizer 104 performs quantization and when the 
electronic watermark data insertion element 305 calcu- 
lates a partial average, and an encoder 313 for encod- 
ing data outputted from the quantizer 311 and 
outputting MPEG data 314 in which the electronic 
watermark data 304 are embedded. 

The electronic watermark data insertion element 

305 includes a partial average calculator 307 for calcu- 
lating a partial average of data outputted from the DCT 
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calculator 303 with reference to the quantization table 
312, a multiplier 308 for multiplying the output of the par- 
tial average calculator 307 by the electronic watermark 
data 304 for individual factors, another multiplier 309 for 
multiplying the data outputted from the multiplier 308 by 
a constant a 306 which is used to vary the size of the 
electronic watermark data to be inserted, and an adder 
310 for adding the data outputted from the DCT calcula- 
tor 303 and the data outputted from the multiplier 309 
for individual factors. The electronic watermark data 
insertion element 305 thus performs processing equiva- 
lent to calculation of the following expression: 

F(i) = f(i) + avg(f(i)) x w(i) x constant 

where f(i) is the data outputted from the DCT calculator 

303, avg(f(i)) is the data outputted from the partial aver- 
age calculator 307, w(i) is the electronic watermark data 

304, the constant is the constant a 306, and i is the 
number of a factor after the data outputted from the 
DCT calculator 303 are zigzag scanned. 

Here, the partial average calculator 307 in the elec- 
tronic watermark data insertion element 305 performs, 
in order to calculate a partial average avg(f(i)) of three 
neighboring points of the data outputted from the DCT 
calculator 303. processing equivalent to calculation of 
the following expressions: 

V(i-1) = (integral part of (f(i-1 )/Q(i-1)) x Q(i-1)| 

V(i) = [integral part of (f(i)/Q(i)) x Q(i)| 

V(i+1) = (integral part of (f(i+1)/Q(i+1)) x Q(i+1)| 

avg(f(i)) = (V(i-1) + V(i) + V(i + 1))/3 

Referring now to FIG. 4, there is shown another 
electronic watermark data detection system to which 
the present invention is applied. The electronic water- 
mark data detection system detects electronic water- 
mark data Inserted by the electronic watermark data 
insertion system described above with reference to FIG. 
3. To this end, it includes a decoder 402 for decoding 
MPEG data 401 which make an object of detection of 
electronic watermark data, a dequantizer 403 for 
dequantizing data outputted from the decoder 402, an 
inverse DCT calculator 404 for performing inverse DCT 
calculation for data outputted from the dequantizer 403 
to obtain image data 405 and outputting the image data 
405, an electronic watermark data extractor 406 for 
extracting, from among the data outputted from the 
dequantizer 403. those data which are supposed to be 
electronic watermark data and outputting the extracted 
data, an adder 409 for adding data in units of an 8 x 8 
pixel block outputted from the electronic watermark data 
extractor 406 for individual factors over one screen, and 
an inner product calculator 410 for calculating inner 
products of data outputted from the adder 409 and elec- 
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tronic watermark data 41 1 to be detected to obtain a 
statistical similarity 412 and outputting the statistical 
similarity 412. 

The electronic watermark data extractor 406 

s includes a partial average calculator 407 for cal- 
culating a partial average of three neighboring 
points of data in units of 8 x 8 pixel block outputted 
from the dequantizer 403 in accordance with 
avg(F(i)) = (|F(i-1)| + |F(i)( + |F(i+1)|)/3 , and a divider 

10 408 for dividing the data F(i) outputted from the dequan- 
tizer 403 by the partial average avg(F(i)) outputted from 
the partial average calculator 407. Here, F(i) is the fre- 
quency component of an 8x8 pixel block outputted from 
the dequantizer 403, and i is the number of each factor 

15 after zigzag scanning of the 8x8 pixel block. 

The electronic watermark data extractor 406 per- 
forms calculation in accordance with the following 
expression: 

20 WO) = F(i)/avg(F(i)) 

where F(i) is the data outputted from the inverse DCT 
calculator 404, avg(F(i)) is the partial average outputted 
from the partial average calculator 407, and W(i) is the 
25 data outputted from the electronic watermark data 
extractor 406 and supposed to include electronic water- 
m?~k data. 

In the following, operation for insertion and detec- 
tion of electronic watermark data of the electronic water- 

30 mark data insertion and detection systems described 
above with reference to FIGS. 3 and 4 is described in 
detail with reference to FIGS. 3 and 4. 

First, operation for insertion of electronic watermark 
data is described. 

35 A current image 301 is extracted for individual 
blocks 302 of 8 x 8 pixels by ordinary MPEG compres- 
sion processing, and the DCT calculator 303 performs 
DCT calculation for the thus extracted data. The elec- 
tronic watermark data insertion element 305 embeds 

40 electronic watermark data 304 into the data of fre- 
quency components obtained by the DCT transform. In 
this instance, the electronic watermark data insertion 
element 305 performs processing equivalent to calcula- 
tion of the following expression: 

45 

F(i) = f (i) + avg(f(0) x w(i) x constant 

where i is the factor number after zigzag scanning of a 
block of 8 x 8 pixels, f(i) is the value of each coefficient 
so obtained by the DCT transform, w(i) is the value of each 
factor of the electronic watermark data 304, and 
avg(f(i)) is the average of absolute values of three 
neighboring points of each factor. Here, avg(f(i)) is cal- 
culated in accordance with the following expressions: 

55 

V(i-1) = (integral part of (f(i-1)/Q(i-1)) x Q(i-1)| 
V(i) = (integral part of (f(i)/Q(i)) x Q(i)| 



EP 0 851 679 A2 



8 



BNSDOCID: <EP 0851679A2_I_> 




15 

V(i+1) = |integral part of (f(i+1)/Q(i+1)) x Q(i+1)| 
avg(f(i)) = (V(i-1) + V(i) + V(i+1))/3 

The quantizer 311 refers to the quantization table s 
312 to perform quantization processing for the data out- 
putted from the electronic watermark data insertion ele- 
ment 305, and the encoder 313 performs encoding of 
the data outputted from the quantizer 311 to produce 
MPEG data 314. io 

Now, operation for detection of electronic water- 
mark data is described. 

First, MPEG data 401 which make an object of 
detection are subject to MPEG decoding processing by 
the decoder 402. The data decoded by the decoder 402 is 
are dequantized by ordinary MPEG decoding process- 
ing by the decoder 402 and inverse DCT calculated by 
the inverse DCT calculator 404 to produce image data 
405. 

The electronic watermark data extractor 406 20 
extracts decoded data from the dequantizer 403, per- 
forms calculation of F(i)/avg(F(i)) and extracts data sup- 
posed to be embedded electronic watermark data. 
Here, F(i) is the value of each factor after zigzag scan- 
ning in units of an 8x8 pixel block outputted from the 25 
dequantizer 403. and avg(F(i» is the partial average 
(|F(i-1)|+ |F(i)|+|F<(i+1)|)/3. The adder 409 calcu- 
lates sum totals of the extracted data in units of an 8x8 
pixel block outputted from the electronic watermark data 
extractor 406 for individual factors over one screen. 30 
After the sum totals of the extracted data for one screen 
are calculated, the inner product calculator 410 calcu- 
lates inner products of the sum totals and the electronic 
watermark data 41 1 to be detected to obtain a statistical 
similarity 412 and outputs the statistical similarity 41 2. 35 

If the statistical similarity 41 2 is higher than a partic- 
ular fixed value, then it is discriminated that electronic 
watermark data equivalent to the electronic watermark 
data 411 are embedded in the MPEG data 401 . 

In this instance, if the electronic watermark data 40 
411 represent inhibition of duplication, then the repro- 
duction apparatus in which the electronic watermark 
data detection system is incorporated can take such a 
countermeasure as to inhibit duplication of the image 
data 405 produced therein. 45 

Having now fully described the invention, it will be 
apparent to one of ordinary skill in the art that many 
changes and modifications can be made thereto without 
departing from the spirit and scope of the invention as 
set forth herein. so 

Claims 

1. An identification data insertion system for digital 
data for inserting electronic watermark data as ss 
identification data having special information into a 
digital image, characterized in that it comprises: 
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a DCT calculator (1 03) for extracting blocks of 8 
x 8 pixels from a current image and performing 
discrete cosine transform calculation for the 
blocks; 

a quantizer (104) for quantizing data outputted 
from said DCT calculator (103); 
a quantization table (105) for being referred to 
when said quantizer (104) performs quantiza- 
tion; 

an electronic watermark data insertion element 
(107) for inserting electronic watermark data 
into the data after the quantization; and 
an encoder (113) for encoding data outputted 
from said electronic watermark data insertion 
element (107). 

2. An identification data insertion system for digital 
data as set forth in claim 1 , characterized in that 
said electronic watermark data insertion element 
(107) includes a partial average calculator (108) for 
calculating an average of absolute values of three 
neighboring points of each factor of the data output- 
ted from said quantizer (104) as a partial average, a 
first multiplier (110) for multiplying the electronic 
watermark data by an output of said partial average 
calculator (108) for the individual factors, a second 
multiplier (1 1 1) for multiplying each factor of an out- 
put of said first multiplier (110) by a constant to be 
used to vary the size of the electronic watermark 
data to be inserted, and an adder (112) for adding 
an output of said second multiplier (111) and the 
output of said quantizer (104) for the individual fac- 
tors. 

3. An identification data detection system for digital 
data, characterized in that it comprises: 

a decoder (202) for decoding compressed 
image data; 

a dequantizer (203) for dequantizing data out- 
putted from said decoder (202); 
an inverse DCT calculator (204) for performing 
inverse discrete cosine transform calculation 
for data outputted from said dequantizer (203) 
and outputting image data obtained by the 
inverse discrete cosine transform calculation; 
an electronic watermark data extractor (206) 
for extracting and outputting, from among the 
data outputted from said decoder (202), those 
data which are supposed to be electronic 
watermark data; 

an adder (209) for adding data in units of a 
block of 8 x 8 pixels outputted from said elec- 
tronic watermark data extractor (206) for indi- 
vidual factors over one screen; and 
an inner product calculator (210) for calculating 
inner products of electronic watermark data to 
be detected and the data outputted from said 
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adder (209) to calculate a statistical similarity 
and outputting the statistical similarity. 

4. An identification data detection system for digital 
data as set forth in claim 3, characterized in that 5 
said electronic watermark data extractor (206) 
includes a partial average calculator (207) for calcu- 
lating an average of absolute values of three neigh- 
boring points of each of data in units of a block of 8 

x 8 pixels outputted from said decoder (202) as a 10 
partial average, and a divider (208) for dividing the 
data outputted from said decoder (202) by the par- 
tial average outputted from said partial average cal- 
culator (207). 

15 

5. An identification data insertion system for digital 
data, characterized in that it comprises: 

a DCT calculator (303) for extracting blocks of 8 
x 8 pixels from a current image and performing 20 
discrete cosine transform calculation for the 
blocks; 

an electronic watermark data insertion element 
(305) for inserting electronic watermark data 
into data after the discrete cosine transform 25 
outputted from said DCT calculator (303); 
a quantizer (31 1) for quantizing data outputted 
from said electronic watermark data insertion 
element (305); 

a quantization table (312) for being referred to 30 
when said quantizer (311) performs quantiza- 
tion and when said electronic watermark data 
insertion element (305) calculates a partial 
average; and 

an encoder (313) for encoding data outputted 35 
from said quantizer (31 1). 

6. An identification data insertion system for digital 
data as set forth in claim 5, characterized in that 
said electronic watermark data insertion element 40 
(305) includes a partial average calculator (307) for 
calculating, using said quantization table (312) 
when a partial average of the data outputted from 
said DCT calculator (303) is to be calculated, an 
average among three neighboring points of abso- 45 
lute values of values obtained each by multiplying 

an integer value obtained by dividing a value of 
each factor outputted from said DCT calculator 
(303) by a corresponding value of said quantization 
table (312) by the corresponding value of said so 
quantization table (312) as a partial average of the 
pertaining factor, a first multiplier (308) for multiply- 
ing an output of said partial average calculator 
(307) by the electronic watermark data for the indi- 
vidual factors, a second multiplier (309) for multiply- 55 
ing data outputted from said first multiplier (308) by 
a constant to be used to vary the size of the elec- 
tronic watermark data to be embedded, and an 
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adder (310) for adding the data outputted from said 
DCT calculator (303) and data outputted from said 
second multiplier (309) for the individual factors. 

7. An identification data detection system for digital 
data, characterized in that it comprises: 

a decoder (402) for decoding compressed 
image data; 

a dequantizer (403) for dequantizing data out- 
putted from said decoder (402); 
an inverse DCT calculator (404) for performing 
inverse discrete cosine transform calculation 
for data outputted from said dequantizer (403) 
and outputting image data obtained by the 
inverse discrete cosine transform calculation; 
an electronic watermark data extractor (406) 
for extracting and outputting, from among the 
data outputted from said dequantizer (403), 
those data which are supposed to be electronic 
watermark data; 

an adder (409) for adding data in units of a 
block of 8 x 8 pixels outputted from said elec- 
tronic watermark data extractor (406) for indi- 
vidual factors over one screen; and 
an inner product calculator (410) for calculating 
nner products of electronic watermark data to 
be detected and the data outputted from said 
adder (409) to calculate a statistical similarity 
and outputting the statistical similarity. 

8. An identification data detection system for digital 
data as set forth in claim 7, characterized in that 
said electronic watermark data extractor (406) 
includes a partial average calculator (407) for calcu- 
lating an average of absolute values of three neigh- 
boring points of each of data in units of a block of 8 
x 8 pixels outputted from said dequantizer (403) as 
a partial average, and a divider (408) for dividing 
the data outputted from said dequantizer (403) by 
the partial average outputted from said partial aver- 
age calculator (407). 

9. An identification data insertion and detection sys- 
tem for digital data, characterized in that it com- 
prises: 

an identification data insertion system of claims 
1 , 2, 5 or 6 and an identification data detection 
system of claims 3, 4, 7, or 8. 
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